ARM unveils two new A.I. computer chip designs

Arm Holdings, the U.K.-based semiconductor design company owned by SoftBank, unveiled two new computer chip designs for artificial intelligence applications on Monday.

One chip, called the Arm Cortex-M55 processor, is the latest in Arm’s line of microcontrollers, the kind of relatively low-cost and power-efficient chips that are used to run sensors and perform simple computational tasks in everything from automobile engines to remote-controlled toys to power tools.

In the past, these sorts of chips generally lacked enough computing power to perform machine-learning functions efficiently. Instead, most of those tasks had to be performed on higher-powered chips, such as Arm’s Cortex-A microprocessors, which are found in most of the world’s smart phones.

The other new chip, called the Arm Ethos-U55 NPU, is designed to run in conjunction with the M55 as a way of accelerating applications that require neural networks, a kind of machine learning loosely-based on the human brain.

Over the past decade, much of the progress in A.I. has come from running these neural networks on graphics processing units, specialized computer chips originally designed to handle the intense number-crunching needed to render images for video games and animation.

But, more recently, a number of companies have created NPUs, or “neural processing units,” chips designed specifically for running neural networks. In benchmark tests, these chips can run neural networks much faster and efficiently than GPUs. Arm claims the Ethos-U55 is the industry’s first “microNPU”—in other words, the first of these neural network-specific chips small enough and power-efficient enough to run on the smallest electronic devices.

Consultants McKinsey & Company forecast that by 2025, AI-related semiconductors could account for $67 billion in annual sales, or about 20% of all computer chip demand. Another consulting firm, Deloitte, has forecast that the market for A.I.-specific chips designed to run on “the edge”—on devices such as mobile phones and smart speakers and not on datacenter-based servers—will total about $2.6 billion this year and double by 2024.

Arm made its first foray into the A.I.-specific chip market in 2018 when it introduced designs for a machine learning processor and another aimed specifically at detecting objects in images. But both of those processors were designed for environments where power efficiency and cost were less critical.

The U.K. firm is not the only chipmaker looking to provide machine learning capabilities designed for low-cost, power-efficient sensors and electronics. Huawei, as part of its push for 5G mobile, has invested in inexpensive NPUs, but it has packaged these as part of larger chips for higher-end mobile phones, such as its Kirin 970 chip. Intel has generally not been active in the edge-computing market, targeting data centers and personal computing devices instead. AMD has created some chips designed to run A.I.-applications on devices such as smart televisions.

Arm, which does not manufacture semiconductors itself but licenses its designs to others, says chips based on these two new designs will ship to customers in 2021. And while the company declined to name customers for the new designs, in the past Arm has licensed its A.I.-specific designs to companies that include Samsung, STMicroelectronics and NXP.

The company says that by yoking the two new chips together as a team, it managed to achieve a 480-times increase in the speed of running machine learning tasks compared to its previous Cortex M chips in benchmark tests. (The first 15-times speed-up comes from the M55, with an additional 32-times improvement from marrying it with the Ethos-U55.) Using the two chips together also results a 25-times improvement in energy efficiency, which is critical for many devices which must depend on battery power. Arm did not say how its new chips perform against competing A.I.-specific ones.

The new chips allow A.I. applications to be brought to areas such as farming, where hundreds or thousands of low-cost sensors equipped with machine learning, can be used to carefully calibrate how much water, fertilizer, or insecticide each plant should receive, Dipti Vachani, ARM’s senior vice president and general manager for automotive and IoT business, says. IoT stands for the Internet of Things, the idea of connecting all kinds of new devices, from security cameras to toasters, to the Internet.

Vachani also says that getting A.I. to function on relatively low-powered devices, rather than having to be in constant communication with cloud-based data centers where most A.I. workloads are run today, is critical for data security and privacy.

For instance, she says smart speakers, such as Alexa or Google Home, could run speech recognition and some machine learning applications directly on the device. Today, those devices have to transmit data through the Internet to remote data centers, raising the concerns about cybersecurity and eavesdropping.

She also says that getting A.I. to work on non-connected, relatively low-powered devices is essential for making connected automobiles and, ultimately, self-driving cars, a reality, and for bringing machine learning to medical devices.

Arm says that the M55 itself will be able to take on machine learning tasks ranging from very simple vibration detection—something even the previous generations of Cortex-M can handle—through to detecting objects in images. When coupled with the Ethos U-55, it can take on higher-level tasks such as detecting specific gestures, deciding whether your fingerprints or your face match the biometrics already stored on the device, and even speech recognition. But more computationally-intensive duties, such as categorizing a wide variety of objects or recognizing faces in real-time from video of a crowd, will still require more power-hungry, and expensive, chips.

Vachani also says that the new chip designs are only intended to perform what is known as A.I. “inference” —which involves taking a data input and using a statistical model that has already been trained to make a prediction from that data. The actual training of the model, the “learning” part of the machine learning, still needs to be done in a data center on more powerful chips.